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variable is the outcome statistically significantly different between 
groups?" The Johnson-Neyman (J-N) technique answers this kind of question in 
analysis of covariance (ANCOVA) settings. In the multi-level modeling 
context, the F-test, which is used in ANCOVA, cannot be applied because the 
assumption of homogeneity of variance within cluster units is violated in 
most cases of data that have multi-level structure. Instead, the approximate 
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Abstract 



One of the innovative insights in the hierarchical linear models (HLM) is to use it 
for “Slopes-as-Outcomes” models. This implies that we consider that the regression 
slopes vary from cluster to cluster randomly as well as systematically with certain 
covariates at the cluster level. Among the covariates, group indicator variables at the 
cluster level, which classify the cluster units into several groups, are often found to be 
significant predictors. If this is the case, the average relationships between the outcome 
and a key independent variable are different from group to group. Then, a question such 
as “at what range of the independent variable the outcome is statistically significantly 
different between groups?” naturally arises. Johnson-Neyman (J-N) technique answers 
this kind of question in the analysis of covariance (ANCOVA) settings. In multilevel 
modeling context, F-test, which is used in ANCOVA, can not be applied because the 
assumption of homogeneity of variance within cluster units is violated in most cases of 
the data that have multilevel structure. Instead, the approximate Walt test can be used to 
determine the region of significance. Mathematica computer software package which is 
capable of symbolic processing allows us to directly obtain the solution. Two examples 
from education and child development are provided in order to illustrate the technique 
and to show how to implement it by Mathematica. 



In hierarchical linear models (HLM), it is conceived that each macro-unit has its 
own regression parameters and that those parameters vary randomly from macro-unit to 
macro-unit. This implies that each macro-unit has different regression slopes. At macro- 
level, we try to explain why there is variability of the dependent variable among macro- 
units using the macro-level independent variables, i.e., information about the 
characteristics belonged to the macro units. Frequently, the information includes 
qualitative variables which creates groups. For example, in the school-effectiveness study 
such as the High School and Beyond Survey (Coleman, Hoffer, & Kilgore, 1982) where 
the data have structure that students are nested within school, suppose one want to study 
the relationship between student’s socioeconomic status (SES) and achievement'. If the 
schools are grouped by the sectors such as private vs. public schools, then each sector has 
its own average relationship. If these intercepts and slopes are different, then it would be 
of interest to ask in which region of student’s SES in fact the two sectors have significant 
difference. 

In growth modeling via hierarchical modeling framework, repeated measures are 
conceived as nested within subjects and thus each subject has his/her own growth 
trajectory characterized by a set of random regression coefficients at level- 1. Thus, those 
random coefficients vary from subject to subject. At level-2, those variability are tried to 
explain by the individual characteristics. Frequently, among those individual 
characteristics categorical variables such as gender, race/ethnicity that characterizes 
individual are included (For example, Huttenlocher et al, 1991; Raudenbush & Chan, 
1993). In this case, we can obtain the average growth trajectory for each group. Then, a 
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question such as “In which age are there statistically significant group differences in 
terms of expected values of the outcome?” can be interesting developmentally. 

Thus, there are many instances that we are interested in the range of the covariates 
where the expected outcome has significant differences among groups in multilevel 
modeling. This is a natural consequence of multilevel modeling because it conceives that 
each macro unit has its own regression lines. And if there are the grouping variables that 
interact with independent variables, then, a question on which region of independent 
variable is actually statistically significant naturally arises. This question can be answered 
by the Johnson-Neyman procedure in Analysis of covariance (ANCOVA) context if the 
linear model is appropriate. Clearly, multilevel model does not satisfy the i.i.d. error 
assumption, the J-N technique can not be applied directly. However, the spirit of J-N 
technique can be applied to the multilevel modeling by changing the test statistic to be 
computed and the subsequently the reference distribution. That is, by casting the J-N 
problem into a linear hypothesis, we can use the Wald statistic in stead of F statistic. 
Though it is not exact as in the F-test in ANCOVA-linear model context, the Wald test is 
a good approximation when the sample size for the cluster units is large enough, as 
freqently the case for the settings to apply the multilevel models. 

Johnson— Neyman Procedure in ANCOVA 

The Johnson-Neyman technique, as it was originally formulated by Johnson and 
Neyman (1936), solves the problems of identifying regions of significance of the 
covariates in analysis of covariance (ANCOVA) when the regression lines are not 
parallel. The idea behind the J-N procedure can be described as follows. If we know the 
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points of the covariates, we can test whether the means for several groups at the points 
are significantly different by constructing a F-statistic and then comparing the value to 
the corresponding critical value of the F-distribution because the test statistic is 
distributed as Fisher’s F distribution if specification of the error distribution, which is 
specified as normal, is correct. The decision rule of rejecting the null hypothesis, such as 
group means are not different at the specified value of the covariates, provides the 
inequality that the value of the test statistic must satisfy in order to reject the null 
hypothesis. Since we don’t know and actually we wish to know the points of the 
covariates, we make the values of the covariates unknown x and then solve the inequality 
with respect to the unknown value of the covariate, x. For the simple design such as two 
groups, i.e., treatment and control groups, and one covarites, a simple formula which 
basically solves a quadratic equation is appealing because it provides the insight on how 
the terms in the formula influence the solution. A relatively simple formula for the 
explicit solution is still available for several groups but only when the number of the 
covariates is one (Huitema, 1980, Chap. 13). If the number of covariates gets two, the 
formula gets complicated (Johnson and Fay, 1950), and if it is more than three, the 
formula is almost intractable. 

Recently, the advent of computational softwares such as Mathematica (Wolfram, 
1999) which explicitly provide the symbolic processing capabilities changes the 
formulation of Johnson-Neyman technique. That is, it can be phrased by general linear 
model framework and the general from of the equation does not have any limitation in 
terms of the number of covariates (Hunka, 1995; Hunka & Leighton, 1997). The 
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disadvantage for this alternative formulation of the J-N technique was that it does not 
provide the immediate solution. Mathematica can overcome this limitation. 



In the linear model, the dependent variable is regressed on the set of independent 
variables which include the qualitative grouping variables and the quantitative covariates. 
Suppose we have g groups and p covariates and we formulate the linear model with 
intercept. Then we have (g-l)p independent variables and 1 intercept. In equation, the 
general linear model can be written as 



where Y is the zzxl vector of observations, and X is a nxP design matrix where 
P = (g - 1) p + 1 p out ofP are continuous covariates and fi is the P x 1 vector of 

parameters, and £ is the nx 1 vector of errors and £ ~ /V(0,cr 2 I) where I is the nxn 
identity matrix. 

Whether the group differences are statistically significant at certain point of 
covariates, we assign the value for the covariates at the point we want to test and create 
contrast matrix K T of order df k xP where rank{K T ) = df k and the hypothesis we test 
can be expressed as: 



The degrees of freedom for the contrast matrix K T is the number of rows which are 
constructed as independent and is df k = g — 1 if we want to compare for all the groups. 



Y = Xl 3 + e, 



( 1 ) 



H 0 : K t (3 = 0. 



( 2 ) 




which will be distributed as 



F-distribution with the numerator degrees of freedom of df k and the denominator degrees 



of freedom of df e . The quantity SS k in the numerator is the sum of squares of the 



contrast and is expressed as: 

SS k =(K T i3) T [K T (X T Xy'K]-'K T j3 (3) 

where ft is the least square estimator of /3 and it is ft - (X T X)~ ] X T Y . The quantity 
SS e is the sums of squares of residual and it is written as SS e = (Y — Y) T ( Y - Y) where 

~ SS 

Y is the predicted value and is written as Y - Xp . The mean squared error MS e = — - is 

d fe 



SS 



the estimator of o . The use of F distribution is justified because — ~ Xdf k under H 0 



SS 

and — f- ~ Xd/ t a °d SS k and SS e are independent. 



SSJdf k 

Thus, the quantity — is compared to the critical value F a . dft . df of the a- 

SS e /df e 



level test: 



ss J Mf L> 

SSJdf e a ' dfk ' df ' 



(4) 



Therefore, for the test of hypothesis implied by A' 7 to be significant at a level the 
following inequality must hold: 

(K T p) T [K T (X T X)-' K]-' K T p-(MS e )(df k F a .' Mt ) > 0 . (5) 

The contrast matrix K T includes the unknown values of covariates. The region of 
significance can be given by solving the above inequality with respect to the unknown 
values. Thus, to obtain the root of the equation directly by solving the equation 

(X r X)-' /if ) = 0 , (6) 
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Mathematica is necessary since the computation requires symbolic algebra. For the case 
of two groups and one covariate, (K T fi) T [K T (X T X)~ ] AT] -1 K T (3 will be ratio of two 
quadratic polynomials. 

If we want to have simultaneous regions of significance, which can be defined as 
a region such that, with confidence 1- a by a a -level test we can state that the groups 
are different simultaneously for all points contained in it for all possible pair of groups 
(Potthoff, 1964), we just replace the constant df k F a . Jf ■ in the above inequality (2) by 

( p + l)(g - l)-F a; ( /)+ o(j, where p is the number of predictor variables. The between 
non-simultaneous and simultaneous region of significance is that the constant part 
df k F M has larger value when we use the simultaneous region than the non- 

simultaneous region of significance. For example, when g = 2 and p = 1 (this is typical 
ANCOVA design where there are control and treatment group and there is a single 
covariate), we use F a . ydf for non-simultaneous case and 2F a l . d ^ for the simultaneous 
case. 

SS k 

Note that we could use a chi-square test based on the fact that the quantity — — 

MS e 

is approximately distributed as a chi-square with degrees of freedom df k , and the 
decision rule for testing H 0 : K T (5 = 0 in (2) is: 

SS 

Reject H 0 if - > x] f . dfk (V) 

where x\=<if k ,a t ^ ie cr it* ca l value for the a level test for the chi-square distribution with 
the degrees of freedom df k . The justification for this can be seen by reminding that 
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— f ~ Xjr and the MLE for a 2 is MS e . However, note that the nice thing about a test 
a ‘ 

based on the F-distribution, unlike the test based on the chi-square distribution just 
mentioned, is that there is no need to know the true value of <J 2 because it cancels out in 
numerator and denominator. Another advantage of the F-test is that this is the exact test 
for any sample size as long as the model is correct. This creates the contrast with the 
approximate chi-square test, which is only exact for the large samples because the test 
relies on the asymptote of MS e . 



Johnson-Neyman type procedure in Hierarchical Linear Model 

As I have described in the context of school effectiveness study and growth 
modelings, there are cases where the Johnson-Neyman type technique is required in 
multilevel model. That is, when the cross-level interaction exists, i.e., interaction between 
macro-level group membership and micro-level covariates 1 , we pose a question: In which 
region of the covarites which group has the statistically significantly higher mean? 

As in the J-N procedure for the linear model, the J-N type procedure in HLM can 
be expressed as the linear hypothesis on the fixed effects. Since the linear hypothesis 
testing is concerned about the fixed effects, the number of levels in the multilevel model 
doesn’t matter. That is, we can apply the same procedure to the three level or higher 
hierarchical models. Here for the purpose of clarity of exposition, I present the two-level 
HLM model, the simplest case. 

1 Of course, the interactions which occur at the same level can be handled by the same way. But the cross- 
level interaction is mentioned here because it is a feature of multilevel modeling and the example scenarios 
were described by this cross-level interaction. 
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A two-level HLM can be written as: 



Let y, , Yj,.., Yj are all independent and 

Yj = A {j y + A 2j Uj +£j,(j = (8) 

where Yj is the x 1 vector of dependent variable; A Xj is the fijXF matrix; A 2j 
is the rij xR matrix; y is the Fxl vector of fixed effects parameters; m ; is the Rx 1 
vector of level-2 random effects; and e ; is the n j x 1 vector of level- 1 random effects. 
The distribution of u ■ ) and £ ■ are assumed to be m ; . ~ N R (0,T ) where T is a R x R 
positive definite symmetric matrix; and e ; ~ iV(0,(7 2 I nj ) where cr 2 ia the level-1 error 
variance and I nj is the n j xn i identity matrix. The design matrix A Xj for the fixed effects 

parameter y involves the qualitative grouping variable at level-2 and the covariates either 
at level- 1 or level-2. 

In ANCOVA we can use F test. But in HLM we need to use the Wald test (Wald, 
1943) to test about the hypothesis on the fixed effects because in HLM the errors are not 
i.i.d., thus we cannot construct F. Since we use the Wald test, the test would be 
asymptotically correct when J goes to infinity. 

Thus, the statistical test for Johnson-Neymann type hypothesis in HLM is a Wald 
test and has general form of linear hypothesis. Suppose we wish to test d K linearly 

independent hypothesis on the fixed effect y . Then using a d K xF contrast matrix K T , 
the null hypothesis can be written as 

H 0 : K J y = 0 (9) 

where rank(K T ) = d K . And the statistic for testing the hypothesis (9) is 
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H=(K T f) T V-'(K T f) 



( 10 ) 



where V K l is the estimator of V K ' and 

V* = Var{K T y) = K T Var(y)K . (11) 

7 is the MLE of y which is 

f=<.’LK v i' A >,y't A !j v i' Y , < 12 > 

;= i n 

where 

Vj=A 2j xA T 2 j +aX ) - (13) 

Then, the variance of y is 

Var(f) = . (14) 

The MLE of Var{y) is obtained by substituting x and o 2 in V j (see (13)) by their 



A A 

respective MLE. We denote it Var(y). Thus V K = K T Var(y)K . Note that 

Var(y) represented in Equation (14) is identical to the Fisher information obtained from 

taking the inverse of negative of the expectation of the second derivative of the log- 

A 

likelihood 2 , and thus Var(y) is the fisher information evaluated at the MLE of a 2 and 



2 The log-likelihood for the model ( 8 ) is 

N 

/ = log[/(7 I a 2 , T, 7 )] = l0g(27T) - \ X log I Vj I . - i jr (Yj - A u y) T V /' (7. - A tj y) and 



thus 



2 ^ 



2 % 



the information with respect to 7 is /(/) = -E[d 2 l /dy 2 ] = 1 A {J .Then, as the sample size 



j = 1 



increases, the MLE of 7 is consistent and asymptotically normally distributed with mean 7 and variance 
[ 7(7) ]’\ which is represented by writing 7 ~ 24A^(7,[/(7)] _1 ) . 
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T . Then the Wald statistic (Wald, 1941) H is constructed as H = ( Ky) T V ( K T y ) as in 
(10), and the H will be distributed asymptotically, i.e., J -> °°, as chi-square with the 
degree of freedom with the number of row in K 7 which was denoted as n K . Thus the 
statistic H is compared to the chi-square critical value to test whether the null hypothesis 
H 0 can be rejected at the specified a -level. Explicitly, we would reject H 0 : K T y = 0 in 

(9) if 

{K T y) T V;\K T y)>x] f -_ dK , a . (15) 

To obtain the region of significance of the covariates, we use the following inequality 

(K T y) T V;\K T y)- X ] f -_ dK , a > 0. ( 16 ) 

and solve for the unknown values in K T . 

It should be noted that the Wald test chosen for the J-N type technique in 
multilevel models corresponds to the alternative J-N technique in linear models which is 
represented by the decision rule (7). This method was also approximate and utilized a 
statistic that is approximately distributed as the chi-square distribution. 

Illustrative Examples 

Example 1. High school & Beyond Survey 

The first example is taken from the High School & Beyond data set. The model 
seeks the relationship between mathematics achievement and the student’s socio- 
economic status. The model was used in Bryk and Raudenbush (1992). 

Briefly speaking, the data are collected in 1984 and this data is the U.S. 
representative sample. There are 7185 students in 160 schools. Out of 160, 70 schools are 
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Catholic and 90 schools are public high schools. The target variable of our interest is the 
student’s mathematics achievement score and the mean for the students in public schools 
is 1 1.4 and that for Catholic schools is 14.2. The difference is 2.8, which is statistically 
significant. This difference is translated into 0.42 units of within-sector standard 
deviation unit (the pooled standard deviation (s.d.) = 6.734). The individual student’s 
SES ranges from -3.76 to 2.69 with the mean of 0 and the standard deviation is 0.78, and 
the school mean SES ranges from -1.19 to .83 with the mean of 0 and the standard 
deviation of .41. Thus those variables have already been centered. In terms of sector 
difference, in Catholic schools the student’s SES ranges from -3.76 to 2.69 with the 
mean of -0.15 and the standard deviation of 0.79, and in public schools, it ranges from - 
2.84 to 1.76 with the mean of 0. 15 and the standard deviation of 0.74. Thus, on average 
the SES is 0.38 (the pooled s.d. = 0.78) higher in Catholic schools than public schools. 
For school SES, it ranges from -1.19 to 0.83 with the mean of 0 and the standard 
deviation of 0.41. Thus, this variable is also already centered. In terms of group 
differences based on the sector, the school SES ranges from -1.19 to 0.69 in public 
schools with the mean of -0.13 and the standard deviation of 0.38, and -0.76 to 0.83 in 
Catholic schools with the mean of 0.17 and the standard deviation of 0.40. Thus also in 
school SES, which can be conceived as a proxy of school resources, Catholic schools are 
more affluent by 0.76 standard deviation unit (0.389 for the pooled s.d.) than public 
schools. 

To summarize, the effect size of the sector on Math achievement is 0.42, 0.38 on 
student’s individual SES, and 0.76 on school SES. According to Cohen’s definition of 
size of the effect size, the first two is considered to be in between small to medium, and 



the third one, the effect size of sector on school SES is considered to be large in social 
science research (Cohen, 1988). Students in Catholic schools do better in mathematics 
achievements than those in public schools; their parents are more educated and richer; 
and the schools are more resourceful. These descriptive statistics are summarized in 
Table 1. 



Insert Table 1 About Here 



The model to describe the relationship between mathematics achievement and 
SES for each sector, that was used in Bryk and Raudenbush (1992), is at level- 1, 

Model 

Level-1: 

iid 

Y tJ = P 0J + P, j (SESy - SES .j ) + By , £, y ~ iV(0, <7 2 ) . (17) 

Thus at level- 1, the model seeks the relationship between mathematics achievement Y-. 

of the student i in school j and the his/her SES relative to the school mean SES. At level- 
2, the intercept f} 0j and the SES slope P tJ vary from school to school depending on the 

school SES and the sector (public or Catholic), and those have random components, 
u 0J and u^j : 

Level-2: 

Poj =Yoo+ 7oi (SES.J- SES..) + y 02 SECTORj + u 0j 

(18) 
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A; -Yio + Y u (SES.j-SES ..) + y i2 SECTORj +u tj , 



where 



fu > 

u 0j 


iid 

~N{ 


f 0 ) 


s 

o 


U { 

V lJ ) 




l T '0 T " J 



; and SECTORj = 1 if the school j is Catholic and 0 if it is 

public; SESjj is a measure of socio-economic status for student i in school j. It should be 

noted that at level- 1 student’s SES was centered around the school (group) mean and at 
level-2 school SES was centered around the grand mean. This specification decomposes 
the SES effect on mathematics achievement directly into within-school and between- 
school portions and thus makes the meanings of the parameters clear in terms of 
contextual effects. Also, the group-mean centering specification of the model at level- 1 
focuses on the relationship between Mathematics achievement and the students SES 
relative to the school mean SES. Thus, this specification creates that the regression 
surface depends on not only student’s SES but also the school mean SES of the school to 
which the student belongs. The school SES, which is computed as the mean of the 
students’ SES in the school, is sometimes interpreted as a proxy measure of the school’s 
resources such as the number of teachers available, facilities, funding, and so forth. 

The above model was fitted by the HLM software (Raudenbush, Bryk, Cheong, 
and Congdon, 2000) to the data via Restricted Maximum Likelihood Method (REML). 
The results are in the Table 2. 



Insert Table 2 About Here 



Note that in Table 2, the school SES was named as MEANSES. 
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Zoo = 12.10 indicates the average predicted score for the students in public school 
whose SES is the school mean and the MEANSES is the grand mean; y l0 = 2.94 

indicates that on average one unit increase in student’s SES increases the math 
achievement by 2.94 controlling for SECTOR and MEANSES. In terms of the sector 
effects, Catholic schools do better by 1.23 ( f 02 = 1.23) when SES and MEANSES are 

controlled, but have lower SES slope (y 12 = -1.64). Thus, Catholic schools do better in 
excellence and provide more egalitarian education. MEANSES works in a way to 
increase both the intercept (y 0l = 5.33), i.e., the mean score given student’s SES and 

SECTOR, and the SES slope (y n = 1.03). Since the mean mathematics achievement gap 
between Catholic and public schools decreases as the individual SES increases, we 
expect that from some point of individual SES to above, the public schools do better than 
Catholic schools. That is, the SES slopes are different for Catholic schools and public 
schools. In this context, naturally a question such as “In what range of SES, private and 
public schools are statistically different and which sector does better in that range?” 
arises and this question is especially important for prospective students and the parents C 
who seek the best possible outcomes. This is the same situation that Johnson-Neyman 
procedure was called for. 

In order to investigate the above question, we first need to know the expected 
regression planes for Catholic and public schools. Those are obtained by taking 
expectations over the distributions of both Sy and Uj ; 

E(X U ) = (Too +Yoi(SES. j -SES..) + y 01 (SECTOR) j } 

( 19 ) 

+ {y 10 + r n (SES.j-SES..) + y n (SECTOR) j }(SES ij -SES.j). 
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Then, the expected achievements for Catholic and public sector are: 
Catholic: 



E(X g ) c = { (Too + 7 o2 ) + 7o. (SES.j - SES .. ) } 

+ { (7,o + 7, 2 ) + 7„ (SES.j- SES.. ) } (SES tj -SES.j ). 

Public: 



( 20 ) 



E(?ij)p = {7oo +Yo\(SES j- SES ..)} 

+ { 7 , 0 + 7, , (SES j- SES .. ) } (SES^ - SES j ). 

The fact that SES.. = 0 simplifies (20) and (21) to: 

Catholic: 



E(Y,j)c ~ ((7oo + 7 o2) + 7o, SES.j } + {(7io + 7i2) + 7n SES.j}(SES ij -SES.j). (22) 

Public: 

E(y„) p = {/oo + 7 0 , SES.J } + (7,0 + 7,i SES.jKSES, -SES.j). (23) 

Replacing the parameters to the estimates provides the predicted values as the function of 
SES tJ and SES.j . 

Catholic: 



Yyic ~ ((7oo +7 o2) + 7oi SES.j} + {(? l0 + f n ) + f u SES.j }(SES jj -SES.j) 
=■ (1 3.32 + 5.33 SES .j ) + (1 .30 + 1 .03 SES j XSES^ - SES .j ). 

Public: 

7, /.|>= l7oo + 7o, SES.j } + (7,o +7,i SES.j}(SES ij -SES.j) 

= (12.10 + 5.33 SES. j) + (2.94 + 1.03 SES. j)(SESy -SES.j) 



16 18 
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The predicted values are on the surface of the student’s SES ( SES i} ) and the school mean 



SES ( SES.j ) axes. We see several sections of the surface by fixing either SES tj or 

SES.j at three different values, i.e., one standard deviation above the mean, mean, and 
one standard deviation below the mean. First, we fix the school mean SES (mean = 0, s.d. 
= 0.41 as in Table 1) at -0.41, 0, and 0.41, and then see the relationship between Math 
achievement and student’s SES (Figure 1). 



Insert Figure 1 About Here 



As can be seen in (22) and (23), once SES.j is fixed, the relationship is linear. 

The box lines represent schools whose school mean SES is one standard deviation above 
the mean, i.e., SES j = 0.41 (denoted as Catholic+ and public+ in the legend); the triangle 

lines represent schools whose school mean SES is the grand mean, i.e., SES.j = 0.00 
(denoted as Catholic and public in the legend); and the diamond-shape lines represent 
schools whose school mean SES is one standard deviation below the mean, i.e., SES.j = - 
0.41 (denoted as Catholic - and public - in the legend). The graph shows that from low to 
middle/high student’s SES, going to Catholic school produces higher math achievement, 
but the gap gets smaller and smaller as the student’s SES increases. After passing the 
certain student’s SES, we expect that going to public schools will be a better choice. The 
value of the student’s SES at which public schools catch up to Catholic schools shifts to 

the right as the school SES increases, i.e., for SES.j = - 0.41, SES jj = 0.34, for SES.j = 0, 



SESjj = 0.75, and for SES j = 0.41, SES (> . = 1.16. This means that the higher the SES of 



the school, the more students receive the benefits of achieving higher mathematics scores 
by attending Catholic schools 3 . 

Though the above description informs us a pattern on how mathematics 
achievement differ in Catholic and public schools as a function of student’s SES given a 
certain school SES, we are not sure whether those differences are in fact statistically 
significant. To answer this question, we need to use a statistical test that was described in 
the previous section. The test, Johnson-Neyman type procedure, determines the sets of 
the pairs of student’s SES and school SES that produce the statistically significant and 
non-significant differences between Catholic and public schools. 

In order to perform the test, we first compute the expected difference between 
Catholic and public schools. For the same student’s SES and school mean SES scores, it 
is obtained from taking the difference between (22) and (23): 

E(Yy ) c -E(Y ij ) p = y 02 + y l2 (SES ij - SES j ) . (26) 

Thus, the expected difference is the linear function of the relative student’s SES score, 
relative to the school mean SES score. The predicted difference is 

Yij’C-Yij,,, =Yo2+Yn(SES ij -SES. J ). (27) 

Replacing the parameter values by their estimates in Table 2 provides the numerical 
relationship: 

3 Of course, we can take a look at the Equations (24) and (25) by the other way around, i.e., mathematics 
achievement as a function of school SES by fixing student’s SES. As we can see, once the student’s SES is 
fixed, Equations (24) and (25) become a quadratic function of school SES. We, however, here restrict our 
attention only on the above case, i.e., mathematics achievement as a function of student’s SES given a 
certain school SES, for the purpose of simplifying the presentation of the idea. 
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(28) 



Y,j ,c -Y ihp = 1 - 23 - 1 M(SES , - SES j ) . 

Thus, the discrepancy decreases as the relative SES increases. 

Let the relative student’s SES be x, i.e., x = p — q, where p is the student’s SES 

and q is the school mean SES. Then, the hypothesis that E{Y ~) c - E{Y {j ) p = 0 can be 
cast into the form of linear hypothesis as: 

H 0 : K T y = 0 (29) 

where K T = (0,0, 1,0,0, x) and y T = (Wo.. YonYio’ Yn.Yn ) • 

Since K T has one row, the degrees of freedom for the reference chi-square 
distribution is one. Thus, the individual SES region of significance is given by solving the 
following inequality with respect to x: 

(K T y) T V- K \K T Y)-x] f __ Xa >0 (30) 

where V K is the estimate of V K in Equation (1 1). If we choose 0.05 level, the critical 
value is 3.84(^j /=l a=005 = 3.84), we have 

(K T y) T V-'(K T f)-3M>0 (31) 

where K T f.= 1.23 — 1.64a: . Since V r is a six by six symmetric matrix and if we represent 



V y by elements, i.e., 












' var(f (x, ) 
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v r 


co v(y,„,y (X) ) 


cov(y l0 ,y 0l ) 


c6v(y 1(l ,y 02 ) 


va?(? 10 ) 


cov(y lt) ,y n ) 


cov(y l() ,f 12 ) 






co v( 7 , ,,/,„,) 


cov(y n ,y 0l ) 


cov(y tl ,y 02 ) 


cdv(y M ,y 1() ) 


var(y,,) 


c6v(y,,,y 12 ) 






cov(f 12 ,y ()0 ) 


cov(y 12 ,y 0l ) 


cov(y l2 ,y 02 ) 


cov(y 12 ,y ln ) 


c6v(y l2 ,y n ) 


vaf(y l2 ) J 


' 




19 



21 



then V K = K T V y K = Var(y 02 ) + 2xcdv(y 02 ,y n ) + x 2 Var(y l2 ) . The actual values for 



V y obtained from HLM is: 



v.. 



3.95 * 10' 2 


1 . 80 * 1 0' 2 


- 4 . 24*1 O ' 2 


2.30 * 10' 3 


1 . 05*1 O ' 3 


- 2.47 * 10~ 3 ^ 


1 . 80 * 1 0' 2 


1 . 36 * 10 '' 


- 4 . 02*1 O ' 2 


1 . 06*1 O ' 3 


8 . 15 * 10' 3 


- 2.41 * 10' 3 


- 4 . 24 * 1 0' 2 


- 4 . 02 * 1 0' 2 


9.38 * 10' 2 


- 2 . 47*1 O ' 3 


- 2.40 * 10" 3 


5.63 * 10' 3 


2 . 30 * 10" 3 


1 . 06 * 10" 3 


- 2 . 47*1 O ' 3 


2 . 47*1 O ' 2 


1 . 33*1 O ' 2 


- 2.65 * 10* 2 


1 . 05 * 1 0' 3 


8.15 * 10 -3 


- 2 . 40*1 O ' 3 


1.33 * 10' 2 


9.15 * 10" 2 


- 2.58 * 10" 2 


- 2 . 47 * 1 0' 3 


- 2.41 * 10' 3 


5.63 * 10' 3 


- 2.65 * 10' 2 


- 2.58 * 10' 2 


5.90 * 10' 2 



The contrast matrix A -7 involves an unknown value x and thus we use 
Mathematica to do the symbolic computation. 



Step 1 : Obtain the estimate of the variance-covariance matrix for y when you run HLM 
by using the keyword “Print variance-covariance”. This command produces a text file 
‘gamvc.daf that contains the variance-covariance matrix of the fixed effects. The matrix 
(33) is the one we obtained. 



Step 2: Define the estimate of the variance-covariance matrix for y , Var(y) : In 
Mathematika, scientific notation is represented as 10 A _. For example, 0.039488440 is 
3.9488440 10 A 002. 
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Vg= {{3.948844010* -002, 1.7959450 10 *-002, -4.2425393 10 *-002, 
2.2987740 10*-003, 1.0537420 10 *-003, -2.474592110*-003} , 
{1.7959450 10*-002, 1.3628002 10 *-001, -4.0245690 10 *-002, 
1.0585086 10*-003, 8.1498270 10 *-003, -2.4135827 10 *-003} , 
{-4.2425393 10*-002, -4.0245690 10 *-002, 9.3802256 10 *-002, 
-2.4743120 10 *-003, -2.4032180 10*- 003, 5.6273247 10 *-003} , 
{2.2987740 10*- 003, 1.0585086 10 *-003, -2.4743120 10 *-003, 
2.4686438 10*- 002, 1.3342234 10 *-002, -2. 6503914 10 *-002}, 
{1.0537420 10*- 003, 8.1498270 10 *-003, -2.4032180 10 *-003, 
1.3342234 10*- 002, 9.1546401 10 *-002, -2.5814267 10 *-002} , 
{-2.4745921 10 *-003, -2.4135827 10*- 003, 5.6273247 10 *-003, 
-2. 6503914 10 *-002, -2.5814267 10 *-002, 5.9003012 10 *-002}} 



Step 3: Define the vector f , the estimate of the fixed effects. 

g= {{12.0950064}, {5.3330565} , {1.2263840} , {2.9377875}, {1.0344270} , {-1.6409540} } 

Note that in order to define the column vector, we enclose each elemets by a blacket { }. 

Step 4: Define the contrast matrix : 

K' = {{0, 0, 1, 0, 0, x}} 



Step 5: Compute the variance-covariance matrix of K T y . 

Vk = K 1 .Vg. Transpose [ K 1 ] 

The output provided by Mathematica for this operation is given below: 
{{0.0938023+ 0.00562732 x+ (0.00562732 + 0.059003 x) x} } 

Mathematica can simplify this result by using the command Simplify. 
Siirplify[VK] 

{ { 0 • 0938023 + 0 . 0112546 x + 0 . 059003 X 2 } } 
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Step 6: compute the statistics H. 

H= Transpose [g] .Transpose[K'] . Inverse [Vy .K' .g 



II 



1.22638 (1.22638- 1. 64095 x) 



0 . 0938023 + 0 . 00562732 x + ( 0 . 00562732 + 0 . 059003 x) x 
1.64095 (1.22638- 1.64095 x) x 
0 . 0938023 + 0 . 00562732 x + ( 0 . 00562732 + 0 . 059003 x) x 



h 



Again, a more concise representation of this equation can be obtained by asking 
for a simplification: 

Sinpl i fy [ H] 

r r 2.69273 (-0.74736 + x) (-0.74736 + x) ,, 
hO. 0938023 7 0 . 0112546 x 7 0 . 059003 x 2 J J . 



As can be seen from the above expression, Mathematica represents 
H ~{K' y) T V~ l ( K ' y) as the ratio of a quadratic polynomial both in the numerator and 
in the denominator. 



Step 7: Solve the equation H -3.84 = 0 for the unknown x. 

Solve [H- 3.84 == 0, x] 

and the results provided by Mathematica are as follows: 

{{xh, 0.359527}, {x^ 1 .29004} } 

for which the two real roots 0.359527 and 1.29004 represent the solution of interest. 

Step 8. Plot the function ( H - 3.84) over the range of - 4 < x < 4 : 

Plot[H- 3.84, {x, -4, 3}, AxesLabel-> {"R. SES", "B-3.84"}] 

where ‘R. SES’ represents the student’s SES relative to the school mean. The plot is 



given in Fig. 2. 



Insert Figure 2 About Here 



The axes labeled as H - 3.84 represents the value of y T KV K ' K T y - 3.84 for 
various values of x, the relative student’s SES. 

The above results analyzed via Mathematica package indicate that significance is 
attained when the groups are equated to a value of the student’s relative SES set to 0.36 
or smallest and 1.29 or greater; i.e., the point at which the function crosses the horizontal 
axis. Significant differences at a < 0.05 will be found for all values of x for which the 
function plotted is above the horizontal axis. With 95 % confidence, we conclude that 
when the student’s relative SES is less than 0.36 the student will be expected to do better 
in Catholic schools than public schools. On the other hand, if the student’s relative SES is 
greater than 1.29, the student will do better in public schools than in Catholic schools. If 
the student’s relative SES is between 0.36 and 1.29, there is no statistically significant 
difference whether student chooses Catholic high school or public high school. 

Another way of examining the student’s relative SES range that produces either 
significant or nonsignificant differences is to directly examine the expected difference 
between Catholic and public schools, represented by Equation (26) or K T y where 

K T = (0,0,l,0,0,x) for x be the student’s relative SES and y T = (7oo,7oi>7o2>7io’7ii’7i2) • 

Since a contrast K T in this case has one row, a 95 % confidence interval on K T y can be 
consructed by a formula 
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K T f±^f K 



(34) 



where 3.84 is critical value at the upper 0.05 cutoff of chi-square variate of one 
degrees of freedom, K T y is the predicted difference as in Equation (27) or Equation 

(28), and V K is the estimate of the Var(K T f ) , which is represented in Equation (11). 
The predicted difference and its confidence interval can be best described by the 

graph. 



Insert Figure 3 About Here 



The bold straight line in Figure 3 represents the predicted difference, expressed by 
Equation (28), or symbolically, by K T f . The upper curve represents the upper bound of 
the 95 % confidence interval and the lower curve represents the corresponding lower 
bound 4 . From the figure, we find that the expected mathematics achievement of students 
in Catholic schools is higher in the region of (-°o, 0.75) in relative student’s SES than 
that in public schools, where 0.75 is the value of the relative student’s SES at which the 
bold line crosses the horizontal axis and this value was the same value that was observed 
in Figure 1, and then the mean achievement of public schools gets higher in the region of 
(0.75, °°) than that of catholic schools. To see if the difference is statistically significant 
or not, we focus on which values the lower bound line and the upper bound line cross the 
abscissa. They are 0.36 for the line of lower bound and 1.29 for the upper bound line. 

4 Note that, strictly speaking, this confidence interval is not simultaneous one. See the comments regarding 
this in the Summary and Discussion section. 
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Thus, we conclude that the expected mathematics achievement of students in Catholic 
schools are statistically significantly higher in the region of (-«>, 0.36) in relative 
student’s SES than that in public schools. On the other hand, the expected mathematics 
achievement of students in public schools are statistically significantly higher in the 
region of (1.29, +°°) in relative student’s SES than that in Catholic schools. 

Out of all 7185 students in our sample, 4993 students (69.5 %) have their relative 
SES lower than 0.36, 2064 students (28.7 %) have their SES between 0.36 and 1.29, and 
128 students (1.8 %) has the relative SES higher than 1.29. Thus, it can be said that, on 
average, about 70 % of the students will gain better math scores if they choose to go to a 
Catholic school rather than a public school when two sectors have the same school SES. 

If we classify those three groups of students by the sector (Table 3), then we find 
that 2469 for Catholic schools and 2524 for public schools out of 4993 students whose 
relative SES is lower than 0.36. Thus, it can be said that 69.6 % (2469/3543) of the 
students in Catholic school made the right decision, and 69.3 % (2524/3642) of the 
students in public schools could have obtained the better math scores if they went to 
Catholic schools instead of public schools. For high relative SES students, 93 students 
(2.6% of the all public school students) in public schools had their relative students SES 
(RSES) greater than or equal to 1.29, and thus they benefited by having gone to public 
schools; 35 students (1.0% of the all Catholic school students) in Catholic schools were 
disadvantaged by having gone to Catholic schools. In overall, 2562 (35.7%) students 
made a right choice in terms of the choice of the sector, and 2559 (35.6%) students could 
have gained better math achievement if they have chosen the other sector of the high 
schools. For the rest of the 2064 (28.7%) students, the sector did not create the 
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statistically significant difference. As a conclusion from the above analysis, we might 
make the following comments if we were asked an advice from the students and the 
parents who were considering the sector of the schools in order to achieve the highest 
possible mathematics scores: Choose a school whose school SES is as high as possible. If 
you find the schools that have the same school SES in both Catholic and public sectors, 
then consider your family SES. Choose a Catholic school if your family SES is lower 
than the school SES or is higher up to 0.36 than the school SES. On the other hand, 
choose a public school if your family SES is extremely higher than the school SES, 
specifically 1.29. If your family SES relative to the school SES is in between 0.36 and 
1.29, then the sector does not matter. For more than two third of the students (69.5 %), it 
would be more advantageous to choose a Catholic sector. 

Though it’s possible to interpret the results directly by the relative student’s SES 
as was done in the above, translating the relative student’s SES back into the original 
scales, i.e., student’s SES and school mean SES, provides an extra insight. Since 
x = p -q and the non-significance region is represented by the inequality, 

0.36 < x < 1 .29 , the information is mapped into the two dimensional space, shown in 
Figure 4. N 



Insert Figure 4 About Here 



The large rectangle delineates the data points in the sample and the two parallel 
lines, q = p — 0.36 and q = p — 1 .29 divides the plane into three regions. The upper left 
area in dark shadow represents the region that Catholic schools do better than public 
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schools significantly. The area that lies between the parallel lines indicates the region that 
there is no statistically significant difference between two sectors in this region. The right 
below area is the region that public schools will produce better results than Catholic 
schools. We can use this figure to answer some interesting questions. For example, let us 
consider a student with certain SES and who is looking for which sector of high school 
he/she should attend in order to achieve better in mathematics. Suppose his/her SES is 
0.5, which is a little lower than the one standard deviation high above the mean (See the 
vertical line af SES = 0.5 in Fig. 4). The answer depends on what level of school SES is 
available. The student does better in public school if he/she goes to a school whose 
school SES is lower than -0.79; there would be no significant difference if the school 
SES is between -0.79 and 0.14; and the student does better in Catholic school if he/she 
goes to a school whose school SES is higher than 0. 14. Of course, to maximize the math 
achievement, higher the school SES, the better his/her math score will be 5 . 

We can use Figure 4 by the other way around. That is, suppose a situation that an 
local administrator of a school district whose high schools have the same school SES 
ponders what kind of students can benefit by going to either Catholic or public sector. 
Assume, for example, the school SES is fixed at the grand mean, 0. Then up to 0.36 of 
student’s SES, those students will do better if they are sent to Catholic schools. There 
will be no statistical difference if the student’s SES is between 0.36 and 1.29. And if 
student’s SES is higher than 1.29, then he/she will make better performance if he/she 
goes to public school. 

5 Since the predicted math achievement is the quadratic function with respect to school SES (see Equations 
(24) and (25) with the coefficient for the quadratic term be negative), there is the school SES that 
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Now, let us consider a slightly different question. Suppose we want to know about 
the range of student’s SES that produces the significant difference between Catholic and 
public sector in terms of mathematics achievement if the two students who have the same 
individual SES are assigned to a typical Catholic school and a typical public school. The 
“typical” indicates that the school SES is the mean SES of all the schools in the same 
sector. Let the individual SES be denoted x, the mean school SES of all the Catholic 
schools be s c , and the mean school SES of all the public schools be s p . In our data, 

those values are s c - 0. 166 and s p - -0. 130 for each. The expected values for those 
students in each typical school are obtained from Equations (22) and (23) by setting 



SES j= s c for Catholic schools and SES.j— s p for public schools: 



E(Ylj)-fy p c a ih. (700 + 7o2 + 7oi 5 c} +{7l0 + 7|2 + T] l^C ) ) 

— (700 (7oi _ 7 io)‘ S C ^"7o2 — 7| l^C ~ Y 12 s C } + (Y 10 + Y\\ S C 7l2 



(34) 



E(Yy )r> 7 >. Pub. ~ (7oo "b Yo\ s p } 4" (7io 4" 7 1 1 }(■* s p ) 

~ (7oo + (7oi ~Yio^p 7n 5 />} + (7io +Yn s p) x 



(35) 



The predicted values for each sector are obtained by replacing the parameter values by 
their estimates and s c =0.1661 and s p - -0.1295: 



’Typ. Cuth. (7o0 +(7oi 7lo)‘ S C *^7o2 Yil S C Y\2 S c) + (-YlO + Yii S C + Yl2^ X 
= 13.963 + 1.649* 



(36) 



and 



= {7oo +(7oi ~YwK ~Yu s p}+(Yw +7, .*/>)* 
= 11.768+2.804* 



(37) 



maximizes the student’s math achievement. For the student whose SES is 0.5, it is 2.20. But there is no 
school whose school SES is 2.20 in our data set. 
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The relationships between the outcome (Math achievement score) and the individual SES 
for each sector are depicted in Figure 5. 



Insert Figure 5 About Here 



From the figure, we see that at about SES = 1.9, the two average lines intersect each 
other. The exact value is 1.900, obtained by equating Equations (36) and (37), and this 
intersection point sifted to the right, compared to all of the three student’s SES values of 
the intersection points (0.34, 0.75, and 1.16) in Figure 1. In the region where SES < 1.9, 
persons in the Catholic schools have higher math achievement on average and in the 
region where SES > 1.9, persons in public schools have higher math achievement than 
the person in Catholic school given the same individual SES. Since there are only two 
students who have SES > 1.9 out of 7185 (0.0003 %), both of whom happened to be in 
public schools, it is clear that, in general, it is more advantageous to go to typical 
Catholic schools than typical public schools in order to obtain higher math achievement. 

Now the question is, “In which region of SES two group means are statistically 
significantly different and which group is higher in that range and in which region of SES 
there is no difference statistically?” In order to answer the question, we use the Johnson- 
Neyman type technique again. As before, we first obtain the expected difference from 
Equation (35) and Equation (36), which is 

ij )ry/l. Culh. — E(Y j j) Tyi , Puh = {(/oi — TioX^C — ■ S />)^‘7o 2 — TllC^C — S P ) — Y\ 2 S C } ^ 9 ) 

+ {/n (s c ~ ^j>) 7i2 }■* 
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Thus the hypothesis that E(Y ~ ^(^Typ.Puh. ~ 0 can be expressed in the form of 
Equation (9), i.e., a linear combination of the fixed effects paramters as: 

H 0 :K T y = 0 

where K T - [0,s c - s p ,l-(s c -s p ),(s c -s p )(x- s c —s p ), x- s c ] and 

I 7 ~ (7oo,7oi > 7o2 > 7io > Tn > 7i 2 ) ■ Since K T has one row, the degrees of freedom for the 

reference chi-square distribution is one. Thus, the individual SES region of significance is 
given by solving the following inequality with respect to x: 

(K T y) T V;\K T Y)-xU, a > 0 (40) 

If we choose 0.05 level, the critical value is 3.84( jj /= , a=005 = 3.84), and in our data, 
since s c =0.1661 and s p - -0.1295, we have 
(K t y) t V~' (K T f) -3.84 >0 where 

K T =[0,0.2956, 1,-0.2956, 0.2956(^-0.0366), x-0.1661]. (41) 

The contrast matrix K ' involves an unknown value x and thus we use 
Mathematica to do the symbolic computation. Following the same steps as in the 
previous computation, we reach the solution for Equation (40): 

x < 1 .07738 or x > 2.63906 . 

The results indicate that the significant region shifts more to the right, higher 
student SES, compared to the case when two sectors have the same school SES. This 
happens because the mean school SES is higher in Catholic schools than public schools, 
i.e., s c = 0.166 and s p - -0.130, and the estimated effects of school SES ( y 01 and y n ) 

are positive both for the intercept ( / 3 0j ) and the student’s SES slope ( /3 I; ). 




30 



32 



The significance is attained when the groups are equated to a value of the 
individual SES set to 1.08 or smallest and 2.64 or greater. With 95 % confidence, we can 
say that when the student’s SES is less than 1.08 the student will be expected to do better 
in Catholic schools than public schools. On the other hand, if the student’s SES is greater 
than 2.64, the student will do better in public schools than in Catholic schools. If the 
student’s SES is between 1.077 and 2.639, whose interval includes 1.90, which is the 
intersection point (Fig. 5), whether student chooses Catholic high school or public high 
school makes no statistically significant difference. Out of all 7185 students, 6572 
students (91.5 %) have their SES lower than 1.077, 612 students (8.5 %) have their SES 
between 1.077 and 2.639, and only one student (0.014 %) has the SES higher than 2.639. 
Thus, it can be said that, based on student’s SES in the sample, more than 90 % of the 
students will gain significantly better mathematics scores in typical Catholic schools than 
in typical public schools. 

Example 2. Early Child Cognitive Development 

In a longitudinal growth study, there are repeated measures for the same subjects. 
In multilevel modeling framework, we conceptualize the problem in a way that each 
individual has his/her own growth trajectory. Suppose at level- 1, a polynomial is 
formulated to model each individual growth on an outcome measure over time. The level- 
1 coefficients are supposed to vary from person to person. At level-2, we frequently try to 
explain those variability by some individual characteristics such as gender, race, and so 
forth. Those individual characteristics that have qualitative nature create the groups 
whose members share the same characteristics. As a result, each group has mean growth 
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trajectory, which differ in terms of the intercepts and age-related slopes. In this context, a 
question such as “In what age range, two groups are statistically different?” may be of the 
interest. Let us consider an example. 

The Greensboro early child cognitive development study (Chirstian, Bachman, & 
Morrison, 2001) aims to study developmental trajectory and schooling effects on 
cognitive ability on young children whose grades ranged from kindergarten to third 
grade. In order to disentangle age-related and schooling-related influences on children’s 
cognitive growth, the study employs “cut-off’ methods, a natural experiment to take 
advantage of school cutoff. Most North American school district designate an arbitrary 
“cutoff’ date such that children whose birthdate precedes the date will be allowed to 
entry to kindergarten or first-grade, whereas children whose birthdate just misses the date 
will be denied entry. The former group of children who entered kindergarten and whose 
age are younger than peers are called “Young” kindergartner and the latter group of 
children are called “Old” kindergartner because their age are relatively older than peers. 
Between the two groups above, there is another group called “Middle” kindergartner 
whose chronological age was between “Young” and “Old”. Other namings such as 
“Young” 1st grader, “Middle” 1st grader, “Old” 1st grader “Young” 2nd grader, 

“Middle” 2nd grader, “Old” 2nd grader, etc. were used in the same way. Thus, the design 
of the study allows us to investigate the net effects of schooling by comparing, for 
example, “Young” 1st grader and “Old” Kindergartner, because “Young” 1st grader 
already received one year schooling and “Old” Kindergartner didn’t receive it though 
they have almost the same chronological age. In terms of age-related growth, we can use 
repeated measures of children who are in the same group. For our analysis, there are 493 
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children and 173 children (35.1 %) are from “Old” group, 187 (37.9 %) are from 



“Middle” group, and 133 (27.0 %) are from “Young” group. In terms of race/ethnicity, 
254 of them were Caucasians and 239 of them are African Americans. Thus, 48.5 % of 
the children are African American and it was oversampled so that we can obtain the 
precise estimates for this group. Also, there are three cohorts depending on the year of 
entry to the study, i.e., 1990-91, 1991-92, and 1992-93 cohorts. Note that “Middle” 
Kindergartner was employed to the study from the 1992-93 study, no observations were 
available to the 1990-91 and 1991-92 cohorts 6 . There were five waves of observations 
and the first observation was made at fall of kindergarten; the second observation was at 
spring of kindergarten; the third was at spring of 1st grade; the fourth was at spring of 
2nd grade; and the fifth was at spring of 3rd grade. At the time of 1998, this study was 
continuing, so the observations for the spring of 3rd grade were not made yet for the 
1992-93 cohorts. Also, there are missing observations because of attrition; at the spring 
of 3rd grade, 39 % had complete observations for 1990-91 and 1991-92 cohorts. The 
descriptive statistics for the key variables are provided in Table 4. 



Insert Table 4 About Here 



In order to study age-related cognitive growth, a two-level hierarchical linear 
model was employed. The level- 1 model was for child j at time; 

6 Since no significant cohort and cohort-grade interaction effects were found, cohort indicators are 



L-l: 



C Math ) ij = /3 0J + P tj (Grade ) .. + p 2J (Grade)] + p 3j ( DFall )„ + £„ , (42) 

iid 

where £ tj ~ N(0,o 2 ) and Math {j is the mathematics score for child j at time i; Grade y is 

the child /s grade at time i and it is coded as —1.5 for fall of kindergarten, -1.0 for spring 
of kindergarten, 0 for spring of 1 st grade, 1 .0 for spring of 2 nd grade, and 2.0 for spring of 
3 rd grade; Grade 2 is the square of Grade tj ; DFall tj is the indicator of fall term 

measurement and thus it is equal to 1 if the math score was measured at fall and 0 if not; 
and £ tJ is the random within-child error. Note that the reason that the indicator of fall 

measurement was included is that we wanted to capture yearly growth trajectory, which 
includes summer term that doesn’t have school, by adjusting the fall of kindergarten 
measurement. 

The level-2 model represents the variability of the level- 1 random regression 
coefficients. 

L-2: 

Poj = Too + 7oi (DBlack) j +y 02 (lQj -7 Q) + y m (LIT j -Llf.) + y 0i (DO) j +y 05 (DM) j 
+ y 06 (MOM ED j - MOM ED ) + u 0j 

Pm =7io +y u (DBlack)j +y n (IQj - IQ ) + y n (DO) j +y u (DM) s +«,, (43) 

Pij =Y 2 Q +'Y 2 \(DBlack).+y 22 (lQ j - IQ ) + y 23 (DO)j +y u (DM) j +u 2J 

P3 j = y 30 

where 



completely dropped from the subsequent analyses. 
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DBlack is the indicator for African American and it is 1 if the child is African American 
and 0 if not; IQ is the child IQ measured at the fall of the kindergarden and it is centered 
around the grand mean; LIT is the child’s home literacy environment measure and it is 
centered around the grand mean; DO is the indicator for the children in the “Old” group; 
DM is the indicator for the children in the “Middle” group ; and MOMED is the 
mother’s years of education and it is centered around the grand mean; represents the 

expected math achievement score at the spring of 1 st grade for the white child in the 
“Young” group whose IQ, home literacy and mother’s years of education are the average; 
y 10 represents the expected slope at the spring of 1 st grade for the white child in the 



“Young” group whose IQ, home literacy and mother’s years of education are at their 
averages; y 20 represents the expected rate of acceleration for the white child in the 
“Young” group whose IQ, home literacy and mother’s years education are at their 
averages; and y 30 represents the mean discrepancy of the fall measurement from the 
general growth trajectory. 

In this model, since there are two groups for race/ethnicity (white and black 
represented by DBLACK variable) and three levels of chronological groups (“Young”, 
“Middle”, and “Old”, represented by two dummy variables “DO” and “DM”), we have 
six trajectories which represent the average growth of mathematics ability from fall of 
kindergarten to spring of 3 rd grade. 

The results for the model in Equations (42) and (43) are in the Table 5. 
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Insert Table 5 About Here 



From Table 5, the average growth trajectories for six groups are all different even 
after controlling for IQ, home literacy environment, and mother’s years of education. 

Now suppose we are interested in the growth trajectories of white Young children 
and black Old children. The expected value for the children in those groups at the average 
IQ, the average home literacy environment, and the average mother’s years education are 
for black Old children: 

E(Y u ) bo =(7oo+7oi +7o 4 ) + (7,0+71, +Yu)(Grade) +(y 20 +y 2l + y a )(Grade)l 

(44) 

+ y 30 (DFall) iJ 

and for white Young children: 

E ( Y ij)wy = (7rn ) + (7,0 )(Grade) t j + (y w )(Grade)] + y 30 (DFall) ij (45) 

Therefore, at the same grade, the expected mean difference is 

E ( Y ij) B o ~ E^wr =(7o, +7o4) + (7u + Y ii ){Grade) ij + (y 2l + y 13 )(Grade) 2 r (46) 

The predicted values can be computed by the following equations for the two 

groups. 

iBO = (7oo + 7 0 i + 7o4 ) + (7,o +7,,+ 7i 3 )(Grade).. + (y 20 + y 21 + y n )(Grade)l 
+ y 30 (DFall) ij 

Y ij W y = y m +y [0 (Grade) iJ +f ll) (Grade)l +y 3{) {DFall) ij (47) 

Replacing the values of the parameters by their estimated values provided in Table 4, we 
obtain the predicted trajectories for the two groups. Those are: 

Y.. bo = 26.463 + S.2S4Grade ij +0.1 42Grade i f. - 1.380DFa//,.. 
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(48) 



Y ijwy = 22. 179 + 10. 1 15 Grade,, + 2.004 Grade] -\3%QDFall . . 

In Figure 6, the two predicted curves are depicted as the function of the grade. 



Insert Figure 6 About Here 



As can be seen from the figure, the two curves are crossing just before the spring 
of the 2nd grade. Before that, the black children in the “old” group had higher math 
scores than white children in the “young” group, but after the spring of 2nd grade, white 
“young” children surpassed the black “old” children. Specifically, at fall of kindergarten, 
black old group is 2.84 higher than white young group and this difference is statistically 
significant (see Table 6). At the spring of kindergarten, the difference increases to 4.25, 
and it stays at the almost same value (4.28) until the spring of 1 st grade. From this point, 
the superiority of black old group starts diminishing. At the spring of the 2 nd grade, the 
difference goes down to 0.52, which is not statistically significant anymore. Finally, at 
the spring of 3 rd grade, white young children do significantly better than black old 
children in mathematics by 6.83, which is the largest margin among the five time points. 



Insert Table 6 About Here 



Though we can test whether there is a statistically significant difference at each 
time point, one at a time, by specifying the appropriate contrast as seen in Table 4, this 
procedure is cumbersome, especially when the data have many time points. If we use the 
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Johnson-Neyman type test developed in this article with the symbolic computation 
capability of Mathematica, we can do all the tests at once. That is, to test the hypothesis 
that E( 1 Y 0 ) bok -E(y v ) mK = 0 at Grade t can be expressed in the form of Equation (9), i.e., 

a linear combination of the fixed effects paramters as: 

H 0 : K T y = 0 (49) 

where K T = [0,l,0,0,l,0,0,0,t,0,t,0,0,? 2 ,0,? 2 ,0,0] and 

— ( v V V V V V V VVV V V V V V V V v ^ 

- V / 00. / 01 » / 02 » / 03» / 04» / 05» / 06’ / 10* / ]] ’ / 12» / 13* / 14* / 20* / 21 » / 22 » / 23’ / 24» / 30/ * 

Since AT r has one row, the degrees of freedom for the reference chi-square distribution is 
one. Thus, the Grade region of significance is given by solving the following inequality 
with respect to t: 

fKV-'K T f-xl-_ ha >0. 

If we choose 0.05 level, the critical value is 3.84(^j /=l a=005 = 3.84), we have 

fKV-'K T f- 3.84>0. 

The same steps can be used to decide the region of significance on grade. In Step 6 
of the previous example, we computed the H =f T KV^ ] K T f statistic. Simplifying the 

expression of H via Mathematica, we obtain 

Siitpl i fy[ H] 

rr 10.4227 (2.08592 + t) (2.08592 + t) (1.21617 - 2 .2056 t+ t 2 ) 

G (2.30517 - 1.73069 t+ t 2 ) (2.06208+ 2.71154 t+ t 2 ) JJ < 

Mathematica represents H as the ratio of a quadraric polynomial both in the 
numerator and in the denominator. In step 7, we solve the equation H - 3.84 = 0 for the 
unknown t. 
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Solve[H- 3.84 == 0, t] 



and the results provided by Mathematica are as follows: 

{{t^ -3.26576} , {t-» -1.71801} , {t ^ 0.519488} , {t -» 1.92323} } 
for which the four real roots -3.27, -1.72, 0.52 and 1.92 represent the solution of interest. 
Plotting the function ( H - 3.84) over the range of - 5 < t < 3 makes clear on which region 
of grade are really significant. 

Plot[H- 3.84, (t, -5, 3}, AxeSLabel-» {"Ooted Grate", "H-3.84"}] 



Insert Figure 7 About Here 



The axes labeled as H - 3.84 represents the value of f T KV K ' K T f - 3.84 for 
various values of t , the child’s Grade. 

The results indicate that significance is attained when the groups are equated to a 
value of the grade set to -3.27 or smallest, between -1.72 and 0.52, and 1.92 or greatest. 
Translating back to original grade, we identify that at the fall of kindergarten, spring of 
kindergarten, spring of I st grade, and spring of 3 rd grade the expected mathematics scores 
of the two groups are statistically different, and on|y at the spring of 2 nd grade, there is no 
statistical difference on mathematics scores between the two groups. 

An interesting observation can be made in Table 6. Comparison the chi-square 
values of spring of kindergarten and spring of I st grade, we find that even though the 
predicted differences are about the same (4.25 vs. 4.28), the chi-squares have the large 
difference (30.79 vs. 1 1.60). Also, at the spring of 3 rd grade, though the predicted 




39 



41 



difference is the largest among five time points (-6.83), the chi-square is rather small 
(4.29) and it marginally achieved the significant (P-value = 0.036). These results were 
occurred mainly because of the attrition of the samples. See Table 4-a) that the sample 
size for the combined group of White Young and Black Old changes 1 19, 1 17, 89, 57, 
and 27 as time goes from fall of kindergarten, spring of kindergarten, spring of 1 st grade, 
spring of 2 nd grade, and spring of 3 rd grade. We can find the sudden drop in sample size 
from spring of kindergarten to spring of 1 st grade, and from spring of 2 nd grade to spring 
of 3 rd grade. 

Actually, K T y in (49) represents the expected difference between White Young 
and Black Old children and K 7 y in H =y T KV^' K 7 y represents the predicted difference 

or we might say the effect size and represents the estimated standard error for the 
K T y , the expected difference. Then, since the estimated difference K 7 y and the 

estimated variance V K are the functions of GRADE, we can see how these changes with 

GRADE by plotting via MATHEMATICA. 

ES= K' .g 

{ { 4 . 28356 - 1 . 8307 t - 1 . 86214 t 2 } } 

where ES, K’, and g represent the effect size K 7 y , the contrast K’, and the estimates of 
the fixed effects y . The letter t in the equation represents the coded grade. The plot of 
this function over the range of - 3 < t < 3 gives Figure 8. 



Insert Figure 8 About Here 
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We can see the value at fall of kindergarten by: 

ESI = ES/. t-» -1.5 

{{ 2 . 8398 }} . 

The symbol “/ . - >” is the MATHEMATICA function that provides the value at t = — 

1.5. Similarly, at other time points, we obtain: 

ES2 = ES/. t-> -1 
{ { 4 . 25212 } } _ 

ES3 = ES/. t-> 0 
{ { 4 . 28356 } } _ 

ES4 = ES/ . t- 1 
{ { 0 . 590728 } } _ 

ES5 = ES/. t-. 2 

{ { - 6 . 82637 } } 

These agree with the values in Table 6. 

Now, since V K is a scalar in this example, we can rewrite H as 

H=fKV-'K T f = ^J^. 

V K 

Then we can investigate which pai't of the equation, i.e., numerator or denominator, 
dominate the value of H, a chi-square test statistic. The investigation is performed in the 
following: 

The plot of the numerator, (K T f ) 2 , the square of the effect size, over the range of 
— 3 < f < 3 is: 
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Insert Figure 9 About Here 



The plot of the denominator, V K , the estimated variance or the square of the estimated 
standard error is obtained by the following steps. 

First, compute the estimated variance. 

V K = K’ .Vg. Transpose [K'l 

A 

where is Var(f) and the numeric values are provided by HLM software. The 

MATHEMATICA output is: 

{ { 1 . 58143 + 0 . 446094 1 - 0 . 346415 1 2 + 

t ( 0 . 237645 + 0 . 289965 1 + 0 . 0709167 1 2 ) + t ( 0 . 20845 + 0 . 294547 1 + 0 . 0922432 t 2 ) + 
t 2 ( - 0 . 175341 + 0 . 0702023 t + 0 . 159011 1 2 ) + t 2 ( -0 . 171074 + 0 . 0929576 1 + 0 . 173681 1 2 ) } } 

We simplify this function as before. 

Siirplify [ Vr] 

{{0.332692 (2.30517-1. 73069 t+t 2 ) (2 .06208 + 2 .71154 t + t 2 ) } } _ 

The plot of V K over the range of — 3 < t < 3 is in Figure 10. 



Insert Figure 10 About Here 



Summary and Discussion 

Johnson-Neyman technique was developed to determine the covariate’s region of 
significance when the parallel slopes assumption does not hold in an ANCOVA context. 
Since, in the ANOCOVA context, the model can be formulated by a linear model, we can 
define the Johnson-Neyman problem by an equation that F-statistic must satisfy, and the 



roots of the equation provide the solution. The practical problem to obtain the roots of the 
equation was solved by using Mathematical capability of symbolic computation. When 
the model is multilevel, unfortunately the F-test, that is exact even for small samples for 
the linear model, is inappropriate because the within group errors are not identical. 
Instead, we can use the approximate Wald test which is asymptotically distributed as a 
chi-square distribution and thus which would be valid when the sample size, especially 
the number of clusters, is large enough. 

In the first example, we illustrated how Johnson-Neyman type technique was 
useful in organizational research where the key covariate is continuous. This example is a 
straightforward extension of Johnson-Neyman technique to multilevel modeling settings. 
Identifying the specific regions of the pairs of individual SES and school SES, which 
produce the statistically significant difference, provides the more precise information on 
which sector of schools is advantageous to achieve the higher math scores. 

In the second example, the Johnson-Neyman type technique was applied to a 
growth model where measurements were made at several discrete ages/grades, though 
those can be theoretically considered as continuous variables. It showed that the 
technique was useful to determine in which time points which group achieves 
significantly higher math scores than the other group, all at once. This all-at-once 
procedure reduces the amount of tasks required to test at each time point. Also, it should 
be noted that the growth curves were quadratic curves, not linear, could be handled by the 
exactly same way as the straight-line case just by changing the contrast matrix in the 
linear hypothesis. 
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Finally, it should be mentioned that, in this article, simultaneous confidence 
approach, as provided by Potthoff (1964, 1983) in the linear model scenario, was not 
considered. The simultaneous confidence approach will be required when we want to 
hold the probability of making a Type I error at a for all statements simultaneously. 
Then, either the Bonferroni or the Scheffe’s multiple comparison procedure should be 
considered. In either way, the simultaneous inference will put a larger critical value of the 
chi-square distribution to protect the test from inflated Type-I error. The detailed study 
will be left to the future study. 
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Figure 1. 



Relationship between Math Achievement and 
Student's SES by Sector (For High, Middle, and 

Low School SES) 
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Figure 2. A Plot of H - 3.84 as the Function of Relative Student SES 
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Figure 3. A 95 % Confidence Interval on the Difference in Math 



Achievement as the function of Relative Student’s SES 

Differaxe (Catholic - Rib li c ) 
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Figure 4. Region of Significant Difference on the Plane of Student’s SES 
and School SES 
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Figure 5. 



Relationship between Math Achievement and 
Student's SES by Sector (typical Schools) 
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Figure 6. 



Figure 6. Mathematics Growth for Black "Old" 
Group and White "Young" Group 
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Figure 7. A Plot of H - 3.84 as the Function of Coded Grade 
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Figure 8. A Plot of the effect size K T y as the Function of Coded Grade 
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Figure 9. A Plot of the squae of the effect size K T f as the Function of 
Coded Grade 
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Figure 10. A Plot of the estimated Variance V K as the Function of Coded 



Grade 




Table 1. Descriptive Statistics for High School & Beyond Data 
Overall: 



Level- 1 Descriptive Statistics 


Variable Name 


N 


Mean 


SD 


Minimum 


Maximum 


MATHACH 


7185 


12.748 


6.878 


-2.832 


24.993 


SES 


7185 


0.000 


0.779 


-3.758 


2.692 


RSES 


7185 


0.000 


0.661 


-3.657 


2.850 



Level-2 Descriptive Statistics 



Variable Name 


N 


Mean 


SD 


Minimum 


Maximum 


MEANSES 


160 


0.000 


0.414 


-1.190 


0.830 


SECTOR 


160 


0.438 


0.498 


0 


1 



Note: 1. SECTOR = 1 if Catholic and = 0 if Public. 

2. RSES is the relative student’s SES, relative to the school mean SES. 



Classified by SECTOR: 

Catholic: 



Level- 1 Descriptive Statistics 


Variable Name 


N 


Mean 


SD 


Minimum 


Maximum 


MATHACH 


3543 


14.170 


6.359 


-2.832 


24.993 


SES 


3543 


0.150 


0.741 


-2.838 


1.762 


RSES 


3543 


0.000 


0.623 


' -3.373 


2.518 



Level-2 Descriptive Statistics 



Variable Name 


N 


Mean 


SD 


Minimum 


Maximum 


MEANSES 


70 


0.166 


0.396 


-0.760 


0.830 



Public: 



Level- 1 Descriptive Statistics 


Variable Name 


N 


Mean 


SD 


Minimum 


Maximum 


MATHACH 


3642 


11.364 


7.080 


-2.832 


24.993 


SES 


3642 


-0.146 


0.788 


-3.758 


2.692 


RSES 


3642 


0.000 


0.696 


-3.657 


2.857 



Level-2 Descriptive Statistics 



Variable Name 


N 


Mean 


SD 


Minimum 


Maximum 


MEANSES 


90 


-0.130 


0.383 


-1.190 


0.690 





Table 2. Effect of Student and School SES on Mathematics Achievement 



The outcome variable is MATHACH 
Final estimation of fixed effects: 



Fixed Effect 


Coefficient 


Standard 

Error 


T-ratio 


Approx . 
d.f . 


P-value 


For INTRCPT1 , / 


h j 










INTRCPT2 , 


7 00 


12.095 


0.199 


60.865 


157 


0.000 


MEANSES, 


r o, 


5.333 


0.369 


14.446 


157 


0.000 


SECTOR, 


702 


1.226 


0.306 


4 . 004 


157 


0.000 


For SES 


slope, / 


hj 










INTRCPT2 , 


7,o 


2.938 


0.157 


18.698 


157 


0.000 


MEANSES, 


7,, 


1.034 


0.303 


3.419 


157 


0.001 


SECTOR, 


7,2 


-1.641 


0.243 


-6.756 


157 


0.000 


Final estimation of variance components: 


Random Effect 




Standard 

Deviation 


Variance 

Component 


df 


Chi-square 


P-value 


INTRCPT1 , 


U Oj 


1.543 


2.380 


157 


605.295 


0.000 


SES slope, Wj i 


0.386 


0.149 


157 


162.309 


0.369 


level-1 , 


£ u 


6.058 


36.703 
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Table 3. Classification of the students by sector and level of their SES 





RSES < 0.36 


0.36 < RSES < 1.29 


1.29 < RSES 


Total 


Catholic 


2469 


1039 


35 


3543 




(69.6%) 


(29.3%) 


(1.0%) 


(100%) 


Public 


2524 


1025 


93 


3642 




(69.3%) 


(28.1%) 


(2.6%) 


(100%) 


Total 


4993 


2064 


128 


7185 
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Table 4. Descriptive Statistics for Early Child Cognitive Development 
Data 

a) Mathematics Score (Outcome Variable) 

Overall: 





N 


| Mean 


Std. Deviation 


Minimum 


Maximum 


MATH1 


493 


11.67 


5.53 


1 


30 


MATH2 


484 


16.44 


6.69 


3 


42 


MATH3 


372 


25.57 


10.11 


4 


58 


MATH4 


284 


38.13 


12.83 


9 


71 


MATH5 


92 


50.86 


13.01 


19 


74 



Classified bv 6 Groups : 



Descriptlves 





N 


Mean 


Std. Deviation 


Minimum 


Maximum 


MATH1 


White_Old 


104 


16.35 


5.84 


6 


30 




White_Middle 


100 


12.95 


4.85 


4 


25 




White. Young 


50 


11.74 


4.69 


1 


25 




Balck.Old 


69 


10.39 


4.91 


1 


28 




Black.Middle 


87 


8.95 


3.94 


3 


23 




Black.Young 


83 


8.14 


3.35 


2 


18 




Total 


493 


11.67 


5.53 


1 


30 


MATH2 


White.Old 


101 


22.18 


6.46 


7 


42 




White.Middle 


100 


18.03 


5.65 


6 


30 




White.Young 


49 


16.39 


5.39 


5 


29 




Balck.Old 


68 


14.57 


5.00 


3 


26 




Black_Middle 


87 


13.75 


6.07 


4 


28 




Black_Young 


79 


11.67 


4.82 


3 


34 




Total 


484 


16.44 


6.69 


3 


42 


MATH3 


White.Old 


95 


32.37 


10.29 


7 


58 




White.Middle 


77 


27.65 


9.00 


9 


55 




White.Young 


40 


26.03 


10.62 


9 


55 




Balck.Old 


49 


22.14 


9.25 


7 


53 




Black.Middle 


53 


20.98 


5.77 


8 


34 




Black.Young 


58 


18.45 


6.23 


4 


39 




Total 


372 


25.57 


10.11 


4 


58 


MATH4 


White.Old 


81 


45.44 


10.62 


20 


71 




White_Middle 


65 


42.66 


12.05 


17 


66 




White.Young 


30 


37.80 


11.84 


19 


64 




Balck.Old 


27 


32.52 


14.38 


9 


61 




Black.Middle 


39 


30.51 


9.68 


15 


59 




Black.Young 


42 


27.93 


7.16 


13 


45 




Total 


284 


38.13 


12.83 


9 


71 


MATH5 


White.Old 


42 


57.71 


8.66 


39 


74 




White.Middle 


0 












White.Young 


14 


54.71 


8.29 


40 


74 




Balck_Old 


13 


41.77 


13.24 


22 


59 




Black.Middle 


0 












Black.Young 


23 


41.13 


13.30 


19 


63 




Total 


92 


50.86 


13.01 


19 


74 




62 



b) Covariates 
Overall: 





N 


Mean 


Std. Deviation 


Minimum 


Maximum 


IQ 


493 


97.50 


15.27 


52 


141 


LIT 


493 


9.96 


3.78 


.00 


17.00 


MOMED 


493 


13.64 


2.39 


6 


23 


DO 


493 


.35 


.48 


0 


1 


DM 


493 


.38 


.49 


0 


1 



Classified by 6 groups : 



Descriptives 





N 


Mean 


Std. Deviation 


Minimum 


Maximum 


IQ 


White_Old 


104 


106.95 


14.85 


72 


134 




White_Middle 


100 


103.90 


12.89 


75 


134 




White_Young 


50 


107.24 


12.61 


82 


141 




Balck.Old 


69 


88.90 


13.16 


52 


126 




Black_Middle 


87 


87.18 


11.04 


53 


121 




Black_Young 


83 


90.05 


10.10 


66 


122 




Total 


493 


97.50 


15.27 


52 


141 


LIT 


White_Old 


104 


12.90 


2.58 


5.00 


17.00 




White_Middle 


100 


12.11 


2.89 


2.00 


17.00 




White_Young 


50 


11.99 


2.26 


6.00 


15.00 




Balck.Old 


69 


7.34 


2.90 


.00 


17.00 




Black_Middle 


87 


6.85 


2.75 


1.18 


15.00 




Black_Young 


83 


7.88 


3.13 


2.61 


16.00 




Total 


493 


9.96 


3.78 


.00 


17.00 


MOMED 


WhiteJDId 


104 


14.93 


2.32 


10 


21 




White_Middle 


100 


14.39 


2.40 


8 


18 




White_Young 


50 


14.02 


2.36 


8 


20 




BalckJDId 


69 


12.48 


2.17 


6 


18 




Black_Middle 


87 


12.70 


1.71 


6 


18 




Black_Young 


83 


12.86 


2.20 


8 


23 




Total 


493 


13.64 


2.39 


6 


23 
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c) HLM Data Set Information 



Level- 1 Descriptive Statistics 


Variable Name 


N 


Mean 


SD 


Minimum 


Maximum 


MATH 


1725 


22.45 


14.43 


1.00 


74.00 


GRADE 


1725 


-0.44 


1.05 


-1.50 


2.00 


GRADES Q 


1725 


1.30 


1.03 


0.00 


4.00 


DFALL 


1725 


0.29 


0.45 


0.00 


1.00 



Level-2 Descriptive Statistics 



Variable Name 


N 


Mean 


SD 


Minimum 


Maximum 


DBLACK 


493 


0.48 


0.50 


0.00 


1.00 


IQ 


493 


97.50 


15.27 


52.00 


141.00 


LIT 


493 


9.96 


3.78 


0.00 


17.00 


MOMED 


493 


13.64 


2.39 


6.00 


23.00 


DO 


493 


0.35 


0.48 


0.00 


1.00 


DM 


493 


0.38 


0.49 


0.00 


1.00 




64 



Table 5. Child Cognitive Development 



The outcome variable is MATH 

Final estimation of fixed effects: 



Fixed Effect 




Coefficient 


Standard 

Error 


T-ratio 


Approx . 
d.f . 


P-value 


For INTRCPTl, 


K 












INTRCPT2 , 


7 00 




22.179 


0.795 


27.903 


486 


0.000 


DBLACK, 


r oi 




- 1.410 


0.825 


- 1.710 


486 


0.087 


IQ / 


y 02 




0.346 


0.026 


13.288 


486 


0.000 


LIT, 


y 03 




0.153 


0.069 


2.208 


486 


0.027 


DO, 


y 04 




5.694 


0.826 


6.894 


486 


0.000 


DM, 


7 05 




3 . 404 


0.834 


4.079 


486 


0.000 


MOMED, 


7o6 




0.215 


0.087 


2.458 


486 


0.014 


For GRADE 


slope, 


Pu 












INTRCPT2 , 


7,o 




10.115 


0.481 


21.034 


488 


0.000 


DBLACK, 


7,, 




- 1.860 


0.486 


- 3.825 


488 


0.000 


IQ , 


7 12 




0.077 


0.015 


5.059 


488 


0.000 


DO, 


7,3 




0.029 


0.481 


0.060 


488 


0.953 


DM, 


7,4 




0.980 


0.527 


1.861 


488 


0.062 


For GRADESQ 


slope, 


Pv 












INTRCPT2 , 


720 




2.004 


0.370 


5.413 


488 


0.000 


DBLACK, 


721 




- 0.811 


0.374 


- 2.169 


488 


0.030 


IQ , 


7 22 




- 0.029 


0.012 


- 2.492 


488 


0.013 


DO, 


723 




- 1.051 


0.354 


- 2.969 


488 


0.003 


DM, 


724 




- 0.157 


0.425 


- 0.370 


488 


0.711 


For DFALL 


slope , 


Pv 












INTRCPT2, 


730 




- 1.380 


0.473 


- 2.915 


1707 


0.004 



Final estimation of variance components: 



Random Effect 




Standard 

Deviation 


Variance 

Component 


df 


Chi-square 


P-value 


INTRCPTl, 


% 


5.416 


29.330 


363 


1135.791 


0.000 


GRADE slope, 


u \j 


2.503 


6.265 


365 


729.976 


0.000 


GRADESQ slope, 


u 2j 


1.275 


1.626 


365 


442.752 


0.003 


level-1, 


£ u 


4 . 437 


19.690 
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Table 6. Predicted Differences between Black “Old” and White 
“Young” Groups at each Grade and their Chi-Square Tests 



Grade 


Code 


Black_01d 


White_Young 


Difference 


Chi-Square 


P-Value 


Fall, K 


-1.5 


12.98 


10.14 


2.84 


13.85 


0.0004 


Spring, K 


-1.0 


18.32 


14.07 


4.25 


30.79 


0.000006 


Spring, 1st 


0.0 


26.46 


22.18 


4.28 


11.60 


0.001 


Spring, 2nd 


1.0 


34.89 


34.30 


0.59 


0.12 


>0.5 


Spring, 3rd 


2.0 


43.60 


50.43 


-6.83 


4.29 


0.036 
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